DOCUK^NT RESUME 



ED 263 170 



TM 8S0 660 



AUTHOR 
TITLE 



INSTITUTION 
SPONS AGENCY 



PUB DATE 
NOTE 
PUB TYPE 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Swarthout, David; And Others 

The Effect of Geographic Area on General Aptitude 
Test Battery Validity and Test Scores* USES Test 
Research Report No. 48. 

Northern Test Development Field Center, Detroit, 
MI. 

Employment and Training Administration (DOL), 

Washington, DC. Div. of Counseling and Test 

Development. 

84 

34p. 

Reports - Research/Technical (143) 
MF01/PC02 Plus Postage. 

^Aptitude Tests; Ethnic Status; ^Geographic Regions; 
Meta Analysis; ^Occupational Tests; Racial 
Differences; Scores; Test Construction; *Test 
Validity; ^Vocational Aptitude 

^General Aptitude Test Battery; Specific Aptitude 
Test Battery; *United States; United States (North); 
United States (South); United States (West) 



ABSTRACT 

Meta-analysis techniques were used to determine 
whether or not there are differences in General Aptitude Test Battery 
(GATB) validity and test scores between samples from the Northern, 
Southern, and Western United States census areas. The sample 
consisted of 26,111 subjects from 122 validation studies of the 
Specific Aptitude Test Battery (SATB). Four different methods were 
xxseC to analyze the differences in validities: (1) comparison of 
avertqe validities, weighted by sample size; (2) comparison of 
average differences in validities, weighted by sample size, between 
geographic areas; (3) comparison of job-family validities; and (4) 
chi-square (Hunter, Schmidt, and Hunter). The overall conclusion was 
that there were no meaningful differences in GATB validity between 
geographic areas. Analysis of mean score differences indicated mean 
geographic area differences on all nine GATB aptitudes: general 
learning ability; verbal, nimerical, and spatial aptitude; form and 
clerical perception; motor coordination; and finger and manual 
dexterity. The Western United States scored highest, with South 
lowest, and North in between. These differences were not attributed 
to racial status, since the same pattern was found for Blacks and 
nonminorities. (Author/GDC) 
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ABSTRACT 



TUnis study used meta-analysis researdi tecimiques to detemine if there are 
differences in General Aptitude Test Battery (GATB) validities and test 
scores between sanples from the Northern, Southern, and Western census areas. 
The saitple consisted of 26,111 subjects fron 122 Specific Aptitude Ttest 
Battery (SATB) validation or revalidation studies analyzed since 1972. 

Pour approaches were used to analyze the differences in validities between 
geographic areas. The first approach carpeured average validities, weigjited 
by sanple size. The largest difference jfounS was .03 coirrelation points. 
The second approach o^pared average differences in validities, weighted by 
sanple size, between geographic areas. The average difference in validities 
for all aptitudes across all area oonparisons was .008 correlation points. 
The third approacJi oonpared job family validities (Hunter, Note 1) and showed 
no significant differences. The fourth af5>roach used the chi~square fron 
Hunter, Schmidt, and Hunter (1979). This analysis showed slightly nore 
significant differences than would be expected ty chance, but this could be 
due to three sources of Type I error bias described by Hunter, et al. (1979) 
that were present in this study. 

The overall conclusion of this study is that there are no meaningful 
differences in GATB validities between geographic areas. 

Mean score differences vsere also analyzed. There were mean differences 
between the geograjhic areas cn al]. nine GATB aptitudes, with the Vfest 
highest. South Icwest and North in between. These differences cannot be 
attributed solely to the differences in racial cdiposition between areas, 
siixje the same pattern of differaices was found for blacks and ncnminorities . 

The results have inplicaticns for the present geographic sairpling requirements 
for SATB research and the transportability of validity. 
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INTRDDOCTION 



The U.S. Biployment Service (USES), in ccxperation vath State enployment 
services, has conducted a continuing program of occupati^al test research 
and developnent since the mid 1930' s. Most of this effort has been dev^oted 
to developing and researching the General /^titude Test Battery (GATE) . The 
GATB consists of 12 tests measuring the following nine vocationally-relevant 
aptitudes: 

General Learning Ability (G) 
Verbal Aptitude (V) 
Hjmerical Aptitude (N) 
Spatial Aptitude (S) 
Ibrm Perception (P) 
Clerical Percept icn (Q) 
Motor C3oordination (K) 
Firmer Dexterity (F) 
Manual Dexterity (M) 

The validation of the GATB for ^ecific occupaticns has resulted in the 
developnent of over 470 Specific Aptitude Test Batteries (SATBs) . These 
batteries consist of oonibinations of two, three, or four GATB aptitudes with 
associated cutting scores. All of the SATBs were developed from enpirical 
research studies. In each study criterion data measuring job proficiency 
were collected along with GATB test scores. Ihe validity of the aptitudes 
was measured by the correlation between ^titude test scores and the criterion. 

One issue that the USES has bee.i concerned with is what variables affect or 
moderate GATB validities. Sccve of the variables that have been postulated 
to moderate test validity are minority group status, sex, geograjJiic area, 
age, education and work experience. The present study looks at one of these 
variables - geographic area. Ihe study uses meta-analysis research techniques 
on SATB validaticn data to determine if there are differences in validities and 
test scores between sanples from the Northern, Southern and Western census 
areas. 

In the early 1970 *s the USES consulted with tiie Equal Enployment Opportunity 
Comnassion (EEX) regarding canpliance of USES tests with the EBOG Guidelines 
on Enployee Selection Procedures (U.S. EEXX:, 1970). EEOC representatives 
questioned the conparability of USES eanples in the North with those in the 
South and Vfest. As a result of those meetings, the USES ^barked on a pnDgram 
to revalidate its SATBs with separate analyses on minority and ncnminority 
groups as well as the three census areas (North, South and West). 

Ihe issue of v^ether geographic area moderates test validity has important 
inplications for the conduct of GATB validity research and is of interest to 
both psychologists vho develop tests and enplcyers are concerned with test 
costs and benefits. Psychologists are interested in the variable of geography 
for tecJinical reasons: (1) Ihe Uniform Guidelines on Enployee Selection Proce- 
dures (U.S. EEOC, 1978) indicate that test transportability is permitted v*ien 
certain conditions are met. In the case of criterion-related validity, sanples 
should be ccmparable in terms of "relevant fectors likely to affect validity 

differences *' (p. 38301). Geograpiiy is one factor thought to affect validity. 

(2) Schppidt and Hunter (1977) have found that for a variety of test types and 
jobs, most or a^l observed evidence for situational specificity is artif actual 
nature. It follows that geography probably has a minimal effect on validity. 



In acJdition to technical issues, the concept of geography moderating validity 
has practical implications: (1) It is important to all large multistate 
employers and those organizations involved in cooperative validity researd. 
because requiring samples with geographic representation greatly increases 
the costs of validity studies. (2) It is of interest to both large and small 
employers who look to the Job Service for test-selected applicants because 
It affects the confidence they can place in GATB validity studies for deve- 
lopment of SATBs done in geographical areas other than those in which applicants 
are tested for job orders. 

Only one citation pertaining to geographic differences was found from compute- 
rized and non-ccmputerized literature searches (Department of Defense, Note 3). 
This study found differences in mean aptitude scores between geograE*iic areas 
for the Armed Forces Qualification Test (APQT) , with youth in the North scoring 
highest and youth in the South scoring lowest. 



SAMPLE 



The sample consists of available data from SATB validation or revalidation 
studies analyzed since 1972. The total N is 26,111 from 122 studies. Oie 
hundred and fourteen stuKSias (N=23,558) used a concurrent validation design and 
eight studies (N=2,553) used a longitudinal design. Table 1 gives minority and 
sex breakdown by geographic area. 

Most of the criterion data consisted of the sum of scores from tvo administra- 
tions of the Standard Descriptive Rating Scale. T!he scale was used to get 
ratings from supervisors on five aspects of job performance (quantity, quality, 
accuracy, job knowledge and job versatility) as well as "all-around" ability, 
however, other types of criterion data were collected and criteria were combined 
in different ways (see Table 2) . The Appendix contains a listing of all 122 
studies and the number of individuals in each geoc/raphic area. 



PROCEDURE 



In each study, validities for the nine GATB aptitudes were computed for each 

geographic area that had more than 25 subjects. Differences in validities 

were analyzed for each area oanparison (North-South, North-West, and South-West) . 



TABLE lA 

Ethnic/Minority Ccnposition of Semple by Geographic Area 





»=25,2aea 








North 


South 


West 




N«10,lll 


N=10, 116 


N=5,061 


Black 


2,648 


4,002 


623 


An\erican Indian 


165 


77 


166 


Asian 


131 


54 


224 


Hiq>anic 


348 


381 


992 


White 


6,819 


5,602 


3,056 


^Infbrnation unavailable for 823. 
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Nuniber of Males and Females in each Geogrs^hic Area 






tf=26,lll 










South ■ 


West 




N=i6,177 


N=10,782 


N=5,152 


Male 


4,814 


5,071 


2,531 


Feinale 


5,363 


5,711 


2,621 
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TAHLE 2 



tXinter of Studies and Subjects for Each Criterion lype 



Type of Stu^ 



t of Studies # of Individuads 



Criterion Measure 



CRl 



CR2 



CR3 



Standard DRS 
Ooncurrent 

TVo criteria 
collected but only 
one used as final 

Multiple Hurdle 



Final criterion 
is ocxrbinaticn 
of different 
criteria 

Final criterion 
is oombinaticn 
of sarne criteria 
(not Standard DRS) 

Longibidinal 
Standard DPS 

Only cne 
criterion 



94 



2 
1 
1 

1 
1 
1 
X 

1 
3 
1 
10 



19,942 



119 

647 
933 
95 

213 
81 
123 
286 

119 
723 
141 
1,902 

81 



270 
107 

329 



Standard DRS 



Special DRS 

Standard DRS 
^>ecial DRS 
Course grades 

Sjpecial DRS 
Standard DRS 
Special DRS 
Mixed Standard 

ODurse grades 

Broad category rating 

Rartcing 

.fecial DRS 

Standard DRS 



122 



26,111 



Mixed Standard 

Rating Scale 
ODurse grades 



Standard DRS 



V^tk Sample 

Cburse grades 
Standard DRS 
Standard DRS 

Ririking 

Performance MDdel 
Standard DRS 
Standard DRS 

Course grades 

Broad category rating 

Ranking 

^>ecial DRS 

Standard DRS 



Sum of CRl and CR2 



CR2 



ODntoination of CRl , CR2 
Oonbination of CRl, CR2 
OcETbination of CRl, CR2 
Oonbination of CRl, CR2 

Sum of CRl and CR2 
Sun of CRl and CR2 
Sum of CRl and CR2 
Sum of CRl and CR2 

Sum of CRl and CR2 



^)ecial DRS 
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Validities were analyzed in two ways for the studies that had nore than 25 
subjects fron each of two or three geographic aree^s. Tl^ere were 74 of these 
studies for Iforth-South; 51 for North-West; and 49 for .Scirth-West, Ihe first 
iDethod for analyzing validities for these studies is tased on the Hunter, 
Schmidt and Hunter (1979) analysis of differential validity* This nethod 
involves ccnputirg the following chi-square for each validity pair. 



v^ere Zi and Z2 are the validity coefficien^.s in Fisher Z form for area 1 

and area 2, and Ni and N2 are saniple sizes for area 1 and 2. Ihe nuniber 

of significant chi-squ:ires was canputed as well as the cumulative chi-square. 

Also, the difference in validities between geographic areas was conputed for 
each of these studies and weighted ty the coiibined sariple size for the two 
areas. The mean of these weighted differences was ocniputed for the nine 
aptitudes for each area ccnparison. 

There were 37 studies that had more than 25 subjects fron just cne geographic 
area. These studies were ociribined with the studies that had nore than 25 
subjects fron each of two or three areas. Validities for this larger da,ta 
set were analyzed in tv;o ways. The first conpared validities weighted ly 
sanple size. Average validities, weighted by sanple size, were oonputed for 
each geographic area and tested for differences. The second analysis conpared 
job family validities (Hunter, Note 1) between geografAiic areas. Hunter 
developed a grouping syston of jobs based on the Data and Things ratings of 
the occupational codes in the Dictionary of Occupational Titles (DOT) 
(United States Department of Labor, 1977) . Each job in the DOT is in one of 
the five job families (see Table 3)- The regression weights for each jcto 
family were used to get predicted criterion scores and the correlation was 
caiputed between predicted and actual criterion scores for each study. 
Average validities, weighted by sanple size, for each of the five job 
families were conputed for each geographic area and critical ratios conputed 
between the areas. 

Mean scores were oOTpared ty oonputing sinple one way analyses of variance 
(ANOVA) cn the total sanple between geograjfSiic areas for sex, age, education, 
plant e:qperience, total experieice and the nine GATB aptitudes. ANOVAs were 
also oonputed separately for blac3c and ncrmiinority subgroups . 




ERIC 



TABLE 3 

Job Family and Test Battery Ocnpoeition 



Contribution to Gonposite 



Job 
Family 



Oognitive Periceptual Psychanotor DOT Data-Things 
GVN SPQ KFM Oode 



1 
2 
3 



59% 
13% 
100% 

73% 
44% 



30% 



11% 
87% 



27% 



56% 



T=0=Setting vp 

T=6=Feeding-0fft»earing 

I>=0=Synthesi2ing 
-l=Coordinating 

I>:2=^alyzing 
=3=Conviling 
=4=0onputing 

I>=5=Cppying 
=6=Catparing 



TfiBLE 4 



Nmiber of Qu Squares Significant at .05 Level 



Nuniber of 

Studies G V N S P Q K F M Total 



Nbrth-South 74 5 3 6 6 11 8 8 13 9 69 



Nbrth-West 51 22 344 744 5 35 



South-V7est 49 23 255 541 0 27 
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Total 



174 9 8 11 15 20 20 16 18 14 131 



136 - 



RESULTS 



Table 4 shews the nuriber of dii-squares significant at the -05 level for 
each area caTparison, When a nantoer of significance tests are performed, 
a certain n\ri>er vould be expected to be significant by chance. Ihat is, 
if a nuirber of sanples were drawn from a population for v*iich it was kxxDwn 
that there are no differences between groqps, for seme of these sanples the 
tests of differences vould be significant. Determining how irany tests of 
differences would be expected to be significant if there are r» differences 
in the population is discussed ty Brozek and Tiede (1952) and Cross and 
Qiaffin (1982). Ihe bincmial probability distribution was used to determine 
the nuriber of tests expected to be significant \jp to 49 studies are 
involved. Jbr 50 or more studies the normal distribution was used to 
determine the nuirtoer of tests expected to be significant. Ihe number of 
chi-squares expected to be significant for the 74 North-South chi-squares is 
7.4; for the 51 Kbrth-West chi-squares it is 5.6; and for the 49 South-West 
chi-squares it is 5.4. Only aptitudes P, C/ K, F and M for North-South and 
aptitude Q for North-West have more chi-squares significant than would be 
e55>ected by chance. 

•teble 5 shows the cumulative chi-squares for each aptitude for each area 
conparison and row and oolmn suns. The cunlative chi-squares for each 
area conparison and total emulative chi-square are significant. The re- 
sults for the total chi-square should be viewed with caution because the 
sanples are not independent. 

Table 6 shews the weighted mean differences in validities for each aptitude 
for each area conparison. The weighted mean difference for all aptitudes 
for all ccnparisons is -.008. These ocnparisons between areas were nade in 
this order - North-South, North-^est and South-Wefit. A minus sign indicates 
that the cDeffld.ent for the second area in the pc^ir was larger. For exanple, 
the mean difference for G between Iforth-South was - .014, which means that on 
the average, the validity coefficient for the South wis .014 higher than the 
North. 

Average validities, veighted by sample size, for each area are shown in Table 
7. These validities include data from studies with more than 25 subjects 
from only one geographic area. Infection of Table 7 shows that for each 
aptitude the validities for the three areas are virtually the same, ihe 
critical ratios between these validities are shown in l^ble 8. Only aptitude 
S between North and South shDwed a significant difference and that difference 
was only .03 correlation points. 

The results for the fourth nethod of analyzing differences in validities 
basea on Hunter (Note 1) are shown in Tables 9-10. Bach study was placed in 
one of the five job families tased on teta and Things ratings of their DOT 
codes. Job families 2 and 3 had only one study each and all the data were 
collected from one gecgraphic area. Ihe critical ratios between areas are 
shown in Table 10. None of the critical ratios were significant. 
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TABLE 5 
Qmulative Chi-Squares 



Nurbar of 

Studies G V N S P Q K F M Ibtal 

North-Scxith 74 86.14 82.0 93.B9 86.99 119.27** 113.38** 109.12** 140.98** 114.66** 946.43** 

North-west 51 56.26 57.17 55.21 54.68 69.88* 77.63** 89.70** 71.18* 66.36 598.07** 

South-Vfest 49 53.08 63.50 50.40 57.64 62.06 66.68* 55.11 45.15 42.97 496.59* 

Ttotal 174 195.48 202.67 199.50 199.31 251.21** 257.69** 253.93** 257.31** 223.99** 2041.09** 



*Significant at .05 level. 
**Significant at .01 level. 
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TABLE 6 

Mean Differences in Validities (Vfeighted by Sanple Size) 



North-South North-West South-West 
N=14,018 N=9,933 ^7 , 859 



G 


-.014 


-.008 


.005 


V 


-.011 


-.007 


.009 


N 


.002 


-.010 


-.021 


S 


-.013 


-.005 


.015 


P 


.010 


-.024 


-.028 


Q 


.016 


.003 


-.039 


K 


-.014 


-.047 


-.008 


F 


,015 


-.014 


-.017 


Z" 


-M012 


-.038 


M007 
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•mBIE 7 



Average Valic3ities 
(Vteighted by Sanple Size) 




North 
N=9,963 


South 
N=10,657 


Vfest 
N=4,804 


G 


.21 


.23 


.22 


V 


.16 


.18 


.16 


N 


.22 


.21 


.22 


S 


.13 


.16 


.14 


p 


16 


.15 


.17 


Q 


.18 


.16 


.16 


K 


.11 


.10 


.11 


F 


.12 


.10 


.10 


M 


.11 


.11 


.12 



•mBlE 8 

Critical Ratios Between Mean Vfei^ted Validities 





G 


V 


N 


S 


P 


Q 


K 


F 


M 


North-South 


-1.51 


-l.OC 


.36 


-2.22* 


1.08 


1.22 


.86 


1.00 


.14 


North-West 


-.34 


.06 


-.34 


-.28 


-.51 


1.25 


-.11 


1.14 


-.51 


South-West 


.86 


.86 


-.63 


1.50 


-1.33 


.29 


-.81 


.35 





'*Significant at .05 level. 
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TABLE 9 



Mean Weighted validities of Job E^sordly Norms 
for Studies in Eadi Job Family by GeograpMc Area 



Job 



Ndnber 
of 



Tbtal 



North 



South 



NLmtoer 
of 



Njffttoer 
of 



Nanber 
of 



West 



Family 


Studies 


N 


validity 


Studies 


N 


Validity 


Studies 


N 


Validity 


Studies 


N 


Validity 


.1 


5 


906 


.15 


3 


244 


.17 


5 


415 


.18 


3 


201 


.07 


2 


1 


126 


.09 








1 


126 


.09 








3 


1 


75 


.28 








1 


75 


.28 








4 


67 


16179 


.24 


55 


6778 


.23 


53 


5529 


.26 


37 


3520 


.24 


5 


48 


8825 


.21 


30 


2941 


.23 


44 


4512 


.20 


19 


1083 


.22 
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TABLE 10 



Critical Ratios Between Mean Vfei^ted Validities 
For Each GeograFhic Area For Each Job Family 



Job Family North-South North-Vfest South-Vfest 

1 -.18 .98 

4 -1.30 -.49 

5 1.25 .15 



1.25 



.62 

-.71 
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The results for the canparison of mean scores are shewn in Tfeible 11. Ihere 
were significant differences betvs^n areas for all variables. The same 
pattern of differences was found for all nine GATE aptitudes - West high. 
South low, and tforth in between. The same pattern of differences vas found 
for blacks and nonminorities (see Tables 12-13) . 

DISCUSSION WTO COtKLUSIOJS 

The primary purpose of this study was to find out if the validity of the 
GATB is the same in the NDrthem, Southern and Western areas of che United 
States, fbur different types of validity analyses were performed. Overall 
the data strongly support the conclusion that there is no difference in 
validity between different areas of the country. 

The two mDst relevant analyses are those presented in Tfeibles 7-8 and 6. 
The data in Tfeibles 7-8 corrpared SATB average validities, weighted by sanple 
size, across all three areas. This analysis used 97% of the total sanple 
v^ile the other two analyses used less than this. The interpretation of 
these results is clear. There are no meaningful differences in area 
validities. The data in Table 6 coopared differences in validities on a 
study-by-study basis, with tlie results averaged over all cotparisons. The 
average difference for all aptitudes across all area conparisons is -.008. 

The above two analyses are the most relevant for another reason. Both 
analyses report results in terms of correlation units. This allows for 
the conparison of the magnitude of differences in validities. Sane of the 
other analyses report only that the differences were greater than expected 
by chance and nothing about the magnitude of the differences in terms of 
validity coefficients. 

Even though the other two validity analyses are not quite as pertinent, 
they still support (though less strongly) the hypothesis of no geographic 
differences. Oie explanation for the slightly weaker su^^rt provided ty 
these two analyses is the existence of three sources of Type I error bias 
that make the .05 level of significance inappropriate. The three sources 
of Type I error bias and their effect on the interpretation of the signi- 
cance level were discussed by Hunter et al. (1979). 

The first source of Type I error bias is the effect of nonnormal test score 
distribution due to selecting applicants using the same test, or a similar 
test, to that used for the validaticxi study. Tb the extent that this happens, 
the result is to increase the probability of falsely finding differences in 
validities beyond the specified .05 level. This particular source of Type 
I error bias did not affect the results of this study very much since most 
of the sanples with workers test selected on instriments similar to the 
GATB were eliminated. 

The seccxd source of Type I error bias results from the lack of oonplete 
independence betv^n validity pairs. Predictor and/or criterion redundancy 
increases Type I error, although it is difficult to calculate the effect. 
This source of bias occurs in the present study because the nine GATB 5ptiti:K3e 
scores (which are correlated with each other) are correlated with the same 
criterion to obtain the nine validity coefficients. Since the present 
analyses were oonposed of three separate carparisons (North-South, North-West, 
and South-West) , each sanple was included twice . 

o 
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TftBLE 11 

Analysis of Variance Results for Ibtal Sanple 



North South West 

»=10, ml N=10, 782^ K=5,3 52^ 





Mean 


SD 


Meaui 


SD 


Mean 


SD 


F 


Sex 


1 .D 


c 
• D 


1. 5 




X.5 


.3 


3.27* 


Age 


32.5 


11.6 


30.5 


10.0 


32.3 


11.1 


94.81** 


Ed 


12.2 


1.7 


11.8 


1.8 


12.4 


1.8 


283.38** 


Pit 


63.1 


73.9 


44.4 


52.3 


46*5 


57.9 


226.86** 


Tot 


82.0 


86.7 


54.6 


63.9 


76.0 


86.8 


325.33** 


G 


97.4 


18.4 


90.8 


18.5 


99.7 


19.2 


531.02** 


V 


98.3 


15.9 


91.7 


15.0 


99.7 


16.4 


660.90** 


N 


96.5 


19.6 


90.3 


19.9 


96.6 


19.8 


317.78** 


S 


99.8 


19.5 


95.9 


19.7 


104.7 


20.0 


355.23** 


P 


106.9 


22.2 


102.1 


21.7 


110.9 


22.5 


306.37** 


Q 


112.6 


18.7 


106.7 


18.0 


113.8 


18.5 


385.84** 


K 


105.4 


19.3 


102.6 


18.8 


105.9 


19.2 


79.13** 


F 


97.8 


22.1 


96.5 


21.1 


99,2 


22.4 


29.96** 


M 


105.3 


23.0 


103.7 


21.1 


107.7 


23.5 


56.14** 



for plant e^^serience is 8786, N for total experi ence is 8908. 
for plant experience is 9469, N for total experi<5nce is 10484. 
9N for plant e;q5erience is 4699, N for total experivaice is 5026. 



*Si9iificant at .05 level. 
**Significant at .01 level. 
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TABIE 12 

Analysis of Variance Results for Blacks 



North South West 

N=2,648a N=4,002^ N=623^ 





Mean 


SD 


Mean 


SD 


Mean 


SD 


P 


Sex 


1.6 


.5 


1.6 


.5 


1.7 


.5 


4.24* 


Age 


32.5 


10.2 


29.3 


8.5 


31.8 


9.4 


101.85** 


Ed 


12.1 


1.5 


11.9 


1.6 


12.5 


1.7 


36.44** 


Pit 


56.7 


59.5 


39.3 


41.7 


45.4 


49.7 


90.72** 


Ibt 


73.0 


70.1 


47.0 


50.2 


67.6 


61.8 


153.46** 


G 


84.4 


15.6 


79.9 


15.1 


84.5 


16.3 


75.21** 


V 


89.1 


13.2 


84.5 


11.7 


88.7 


12.9 


120.59** 


N 


84.3 


18.5 


79.9 


18.3 


83.6 


18.6 


47.00** 


S 


89.6 


17.5 


86.9 


17.0 


90.5 


17.9 


25.66** 


P 


96.7 


22.3 


95.5 


21.2 


99.5 


22.3 


9.93** 


Q 


105.0 


18.8 


102.4 


17.0 


105.8 


18.1 


22.86** 


K 


102.9 


20.2 


102.2 


19.2 


102.7 


18.7 


1.05 


F 


91.6 


21.7 


93.1 


21.0 


92.3 


21.0 


4.01* 


M 


99.6 


22.0 


102.2 


20.5 


101.4 


21.8 


11.87** 



for plant experience is 2424, N for total experience is 2440. 
^ for plant experience is 3750, N for total experience is 3874. 
^ for plant experience is 611, N for total experiaice is 619. 



♦significant at .05 level. 
^^Significant at .01 level. 
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mSIE 13 

Analysis of Variance Results for NbtrniDorities 



Iforth South West 

S=6,819^ N=5,602fa N=3,056C 





Mean 


SD 


Mean 


SD 


Mean 


SD 


P 


Sex 


1.5 


.5 


1.5 


.5 


1.5 


.5 


.48 


Age 


32.6 


12.2 


31.8 


10.9 


32.4 


11.5 


8.39** 


Bel 


12.3 


1.8 


11.7 


1.9 


12.6 


1.7 


274.12** 


Pit 


67.4 


80.2 


48.0 


58.4 


45.0 


59.3 


151.27** 


Tbt 


87.3 


93.6 


61.7 


71.7 


80.9 


94.3 


130.51** 


G 


102.8 


16.6 


97.4 


17.4 


106.0 


16.8 


294. IS** 


V 


102.4 


15.1 


96.6 


15.1 


104.9 


15.3 


369.33** 


N 


101.6 


17.7 


%.8 


18.3 


102.1 


17.5 


138.49** 


S 


103.9 


18.7 


100.8 


19.1 


109. 2 


18.8 


196.56** 


P 


111.1 


21.0 


106.9 


21.2 


115.4 


20.9 


168.67** 


Q 


116.0 


17.7 


111.5 


17.3 


117.3 


17.7 


145.43** 


K 


106.2 


18,9 


103.4 


18.1 


106.6 


18.8 


43.66** 


F 


99.8 


22.0 


^.1 


21.3 


100.0 


22.4 


11.86** 


M 


107.1 


23.2 


104.3 


21.7 


108.5 


23.7 


39.79** 



aN for plant e55)erience is 5824, N for total experieice is 5883. 
^ for plant experience is 5218, N for total experieice is 5439. 
for plant experience is 2781, N for total ejqperience is 2980. 



♦significant at .05 level, 
♦♦significant at .01 level. 
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The third source of Type I error bias is due to differential range restriction 
on the predictor variables. This has the effect of showing a statistically 
significant difference between two sec that would be statistically 
significant if the extent of range resL.xction had been equal on both variables. 
Thus^ in scene cases, this results in falsely rejecting the null hypothesis 
and so increases the "real" alpha level to scfne value above the assigned .05 
level. There is some differential range restriction of predictor scores 
across the different geographic regions but in this stidy the overall 
increase in Type I error due to this source of bias is prob^ly not great. 

Because of the very large sample size used in this study, (N=26,lll) the 
probability of detecting true differences when they really exist (statistical 
power) is extremely high. An examination of the results of this study shows 
that most of the statistically significant differences between areas were 
small in magnitude and were detected due to high statistical power associated 
with this sanple. 

The impact of having high statistical power and increased Type I error bias is 
as follows: When we failed to detect differences in validity between areas, we 
are quite sure that such differences really do not exist because of the high 
power of our test. But, v*»en we find significant differences between areas, 
we cannot be sure if they are real differences, or were detected due to the 
Type I error biases. Also, even if we wers sure that such differences were 
true statistically significant differences, they are too small to be meaningful 
and practically useful. 

It is clear that there are differences between geographic areas in mean 
aptitude scores. These differences cannot be attributed solely to the 
differences in racial composition between areas, since the same pattern of 
differences was found for blacks and nonminorities. One explanation may be 
that areas differ in factors such as urban-rural composition, quality of 
education and socio-economic status that Tyler (1965) found to affect 
aptitude scores. 

The theoretical implications of this study support the contention of Schmidt 
and Hunter (1977) that most observed variance of validities is artif actual. 
That is, validities are more stable than previously thought, and are trans- 
portable across such diverse variables as job grouping, race, and geographic 
regions. 

The practical implications of this study for the USES testing program are 
twofold. First, since there are no meaningful differences in validities 
between different geographic areas, the requirements for regional sampling for 
SATE Test Development Projects should be eliminated. This would reduce the 
time and effort involved in developing SATBs. Second, since there is no 
evidence of differences between areas on job family validities, the concern 
over geography as a moderator of validity, would not be a barrier to expanding 
the North Carolina Validity Generalization Project to otiier parts of the 
country. 
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APPQOIX 



DOT Title and Code and Nuriber of Subjects fran Each Geographic Area 

Nurnber of Subjects 



or 


DOT 

Tl t-l A 


DOT 
Code 


Total 


North 


South 


West 


9 


o^ocK cierK 


OOO "iOI i^tfO 

2^2.387-058 


151 


55 


77 


19 


4 


o^wii^^ rjadiine ^^eLocwL 


•JOT CQO A>IC 

/o/ . do2-U4d 


2C3 


25 


132 


51 


7 




A*5Q *5<i1 m A 

uz^^•zol-ulu 


95 


53 


33 


9 


Q 


V^CllULCLL Wrf^AZLCIUwL 


OOC >l C5 AT A 


102 


13 


79 


10 


in 


o La pile L 


202.362-014 


622 


415 


76 


131 


10 




'5A'7 QCO—ATA 

ZUj. Joz-UlU 


431 


226 


87 


118 


in 


TV/Til c#- 


•50*3 COO f\CC 

2Uj.5o2-0dd 


141 


108 


22 


11 


1 1 


L Ut= L 


OCA OOI AOO 


154 


105 


16 


33 


11 

JUL 


cpen uer 


fi^f\ OOT rtOO 

860.381-022 


119 


46 


73 


0 


1 9 


nacninisu 


^rtrt OOrt rtOO 

600.280-022 


283 


154 


45 


84 




iracKoger f nana 


920.587-018 


445 


376 


33 


36 




LXiecKer xx 


id09. 687-010 


121 


80 


0 


41 


"^1 


LnecK ,L 11 


209.687-010 


59 


59 


0 


0 




Bindery Worker 


653.685-010 


185 


133 


48 


4 


38 


File Clerk II 


206.367-014 


2U 


85 


126 


0 


43 


Automobile Mechanic 


620.261-010 


425 


144 


148 


133 


44 


Punch-Press Operator I 


615.482-022 


89 


34 


42 


13 


45 


Shipfitter 


806.381-046 


252 


82 


122 


48 


47 


Nursery School Attendant 


359.677-018 


174 


57 


66 


51 
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SATB Kb. 

Study llo. 


LAJL 

Title 


DOT 
Oode 




l*itiDer of Subiecta 




Itotal 


North 


South 


West 


53 


Spixmec, frame 


682.685-010 


180 


11 


169 


0 


61 


Pluntoer 


862.381-030 


253 


100 


118 


35 


61 


Pluniser 


862.381-030 


411 


0 


411 


0 


63 


Garment Folder 


789.687-066 


103 


10 


93 


0 


68 


Refinery Operator 


549.260-010 


194 


63 


90 


41 


71 


OoemetologLst 


332.271-010 


386 


0 


0 


386 


72 


Electrician 


824.261-010 


394 


SO 


161 


183 


72 


Electrician 


824.261-010 


253 


0 


253 


0 


74 


Central-Office Repairer 


822.281-014 


142 


30 


103 


9 


80 


Radiologic Teciixologist 


078.362-026 


137 


14 


41 


82 


82 


Sheet-Metal Vforker 


804.281-010 


152 


91 


61 


0 


101 


Assembler, Autcrobile 


806.684-010 


213 


119 


94 


0 


115 


Weaver 


683.682-038 


126 


19 


96 


9 


120 


Fire Fighter 


373.364-010 


130 


0 


130 


0 


124 


Tractor-Trailer-Trudc Driver 


904.383-010 


320 


97 


114 


109 


126 


Vfelder, Ocntoinaticn 


819.384-010 


220 


56 


38 


126 


131 


Industrial-Tcuck Operator 


921.683-010 


202 


107 


69 


26 


135 


Producticxi-Madune Tender 


609.685-010 


227 


96 


113 


18 


141 


■Bench Assaiibler 


706.684-642 


160 


18 


142 


0 


144 


Madiinist, Mbod 


669.380-014 


100 


27 


52 


21 


145 


Ciashier-Checker 


211.462-014 


119 


0 


28 


91 
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SATO No. 

or DOT 
Study No> Title 



DOT 
Code 



Number of Subjects 

Total ^North South West 



153 


loom Fixer 


683.260-018 


156 


16 


140 


0 


165 


Packager, Hand 


920.587-018 


102 


32 


70 


0 


168 


Yam-Texturirq-Hachine Operator 


589.685-102 


111 


21 


90 


0 


177 


Millwright 


{538. 281-018 


302 


165 


113 


24 


180 


Keyptnch Operator 


203.582-030 


353 


194 


131 


28 


182 


Laborer, Stores 


922.637-058 


127 


30 


64 


33 


199 


Audit Clerk 


210.382-010 


300 


218 


18 


64 


200 


Reservations Agent 


238.367-018 


310 


^0 


40 


57 


207 


Welder, Arc 


v"10. 384-014 


162 


27 


126 


9 


211 


Welder, Arc 


810.384-014 


461 


305 


118 


38 


211 


Welder, Arc 


810.384-014 


81 


81 


0 


0 


217 


Proof-Machine Operator 


217.382-010 


243 


43 


89 


111 


220 


Coil Winder 


724.684-026 


115 


64 


51 


0 


228 


Injection-Molding-Hacdiine Tender 


556.685-038 


155 


38 


70 


47 


231 


Surgical Technician 


079.374-022 


250 


134 


102 


14 


235 


Metal Fabricator 


619.360-014 


173 


9 


88 


76 


236 


Police Officer I 


375.263-014 


121 


0 


121 


0 


238 


Cook 


313.361-014 


114 


34 


62 


18 


239 


Ward Clerk 


245.362-014 


185 


52 


74 


59 


259 


Teller 


211.362-018 


291 


106 


77 


108 
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SWB No. 

or DOT 
Study No. Title 



DOT 
Code 



Number of Subjects 

Total North South West 





ULotL^fiL f v«JLV JLJL 

Drafter, Geological 
ULTaJLucL, necnaiiicaj. 
Drafter, Structural 


010.281-018 

AA"1 OOl AT A 

005.281-014 


32o 


39 


36 


251 


267 


Tire Builder, Automobile 


750.384-010 


239 


95 


53 


91 


270 


Nurse, Licensed Practical 


079.374-014 


204 


81 


108 


15 


274 


Fbod-Service Worker, Hospital 


355.677-010 


170 


41 


81 


48 


276 


Salesperson, General Merchandise 


279.357-054 


171 


89 


82 


0 


278 


Sales Clerk 


290.477-014 


163 


48 


72 


43 


280 


Structural-Steel Worker 


801.361-014 


249 


185 


10 


54 


281 


Electronics Assembler 


726.684-018 


56 


24 


32 


0 


282 


Nurse Aide 


355.674-014 


136 


0 


136 


0 


286 


Computer Operator 


213.362-010 


213 


91 


38 


84 


287 


Psychiatric Aide 


355.377-014 


334 


182 


81 


71 


309 


Proof-Machine Operator 


217.382-010 


172 


94 


78 


0 


310 


Electronics Assembler 


726.684-010 


185 


4*; 


71 


68 


313 


Automobile-Body Repairer 


807.381-010 


107 


47 


3 


57 


326 


Respiratory Therapist 


C79.361-010 


496 


281 


76 


139 


327 


Psychiatric Technician 


079.367-022 


384 


101 


154 


129 


329 


Administrative Clerk 


219.362-010 


407 


135 


172 


100 


330 


Oiemical Operator III 


559.382-018 


62 


56 


0 


6 
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SATB No. 
or 

Study No. 


DOT 
Title 


DOT 
Code 


Total 


Nurtber of Subjects 
North South 


West 


332 


Ifotel deck 


238.362-010 


406 


228 


101 


77 


335 


Extruding-Machine Operator 


691.382-010 


142 


26 


79 


37 


336 


Knitting-Machine Operator 


685.665-014 


209 


86 


123 


0 


342 


Water-^eatment-Plant Operator 


954.382-014 


222 


147 


38 


37 


343 


Operating Engineer 


859.683-010 


90 


0 


0 


90 


343 


Operating Engineer 


859.683-010 


270 


128 




102 


360 


Yam Winder 


681.685-154 


207 


0 


207 


0 


363 


Maintenance Repairer, Factory or Mill 


899.281-014 


233 


85 


9^ 


59 


370 


Maintenance Mechanic 


638.281-014 


141 


117 


10 


14 


375 


Lather 


842.361-010 


114 


17 




72 


376 


Mailing-Machine Operator 


208.462-010 


128 


45 


78 


5 


379 


Transportation Agent 


912.367-014 


131 


120 


11 

JbJb 


0 


381 


Electronics Assembler 


72€. 684-018 


100 


7 




4 


393 


Hospital-Admitting Clerk 


205.362-018 


178 


78 


78 


22 


398 


Teacher Aide II 


249.367-074 


266 


95 


153 


18 


407 


Ojality Control Technician 


529.387-030 


152 


41 


10 


101 


414 


Assembler/ Electrical Accessories I 


729.687-010 


191 


116 


52 


23 


427 


^pooler Operator, Autcxnatic 


681.686-018 


126 


0 


126 


0 


436 


Fbcd-Service Worker, hospital 


355.677-010 


127 


58 


26 


43 


447 


Welder/ Production Line 


819.684-010 


177 


119 


49 


9 
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SkTQ No , 

or DOT 
Study No, Title 



DOT 
Code 



Number of Subjects 



Total North South West 



456 


Assembler, Eitiall Products 


739.687-026 


183 


50 


114 


19 


465 


covering-Machine Operator 


681.685-038 


65 


0 


65 


0 


466 


Material Handler 


929.687-030 


44 


0 


44 


0 


467 


Electronics Assembler 


726.684-018 


276 


76 


153 


47 


468 


Cigarette Ini^jector 


529.567-010 


64 


0 


64 


0 


469 


Chemical Operator II 


558.5B5-014 


246 


84 


149 


13 


470 


Weaver 


769.684-050 


81 


0 


81 


0 


471 


Electronics Inspector 


726.684-022 


644 


0 


644 


0 


472 


Appliance Assembler, Line 


827.684-010 


107 


0 


107 


0 


473 


Gambling Dealer 


343.467-018 


933 


933 


0 


0 


473 


Gambling Dealer 


343.467-018 


123 


0 


0 


123 


1001 


Central-Suf^ly Worker 


381.687-010 


431 


218 


128 


85 


1002 


Data lypist 


203.582-022 


174 


83 


34 


57 


1003 


Etched-Circuit Processor 


590.684-018 


258 


105 


121 


32 


1004 


Cytotechr legist 


078.281-010 


131 


65 


39 


27 


1005 


Assembler 


723.684-010 


91 


10 


81 


0 


1006 


Machine Operator II 


619.685-062 


247 


132 


103 


12 


1007 


Supervisor 


529.137-026 


75 


0 


75 


0 


1008 


Power-Reactor Operator 


952.362-022 


329 


0 


329 


0 


1009 


Customer-Service Representative 


239.367-010 


278 


132 


90 


56 



SATB Nd. 

or DOT 
Study No, Title 



DOT 
Cede 



Total North South West 



1010 








65 


145 


76 


1011 


Packager, Hand 


920.587-018 


203 


41 


40 


122 


1012 


Ehvirorxnental-Oontrol-System- 
Installer-Servicer 


637.261-014 


262 


0 


262 


0 


3048 


Pipe Fitter 


862.261-010 


95 


95 


0 


0 



- 25 - 

34 

ERIC 



